Material for “ Application of the Bayesian MMSE Estimator for Classification Error to Gene - Expression Microarray Data ” Lori

The introduction of our paper discusses the leave-one-out and cross-validation error estimators. In our implementation of cross-validation, we use k = 5 folds and 5 repetitions, each with different partitions. The basic bootstrap zero estimator, ε̂b0, [3], [4] generates B bootstrap samples, each consisting of n equally-likely draws with replacement from the original sample of size n. Each bootstrap sample is then used to design a surrogate classifier, and the points left out of the bootstrap sample are used as holdout to estimate the error of the surrogate classifier. The bootstrap zero estimator is the average of these errors. Like cross-validation, this error estimator is randomized because of the randomly selected bootstrap samples and tends to be pessimistic because the expected bootstrap sample size is only 0.632n. In our simulations, we use the popular .632 bootstrap error estimator with B = 100, which attempts to correct the pessimistic bias of the bootstrap zero estimator with optimistically biased resubstitution. In particular,